Papers with masked models
Garden Path Recovery in Causal and Masked Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | a linguistics study of garden-path sentences shows that recovery dynamics are important for linguistic evaluation . causal models show larger within-model disambiguation effects than masked models overall . |
| Approach: | They propose to compare garden-path recovery in causal and masked language models . they use 100 English garden- path/control pairs spanning three constructions . |
| Outcome: | The proposed model shows that decoder-only models exhibit sharper disruption at the point of syntactic revision, while encoders appear comparatively buffered at the disambiguator due to right-context access. |
Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a critical task in information extraction that is not covered in recent benchmarks. |
| Approach: | They compare 13 auto-regressive models using prompting and 16 masked models using fine-tuning on 14 NER datasets covering English, French and Spanish. |
| Outcome: | The proposed models outperform auto-regressive models in English, French and Spanish on 14 NER datasets. |
Using Shapley interactions to understand how models use structure (2025.acl-long)
Copied to clipboard
| Challenge: | Language models are intricately structured systems, and attribution measures are important for understanding how they combine features to influence outputs. |
| Approach: | They use Shapley Taylor interaction indices to examine how language and speech models internally relate and structure their inputs. |
| Outcome: | The proposed methods show that language models encode phonetic interactions . they show that the inputs are more entangled for pairs where a consonant influences a vowel or approximant . |